Generating templated documents using machine learning techniques

ABSTRACT

Systems and methods of predicting documentation associated with an encounter between attendees are provided. For instance, attendee data indicative of one or more previous visit notes associated with a first attendee can be obtained. The attendee data can be inputted into a machine-learned note prediction model that includes a neural network. The neural network can generate one or more context vectors descriptive of the attendee data. Data indicative of a predicted visit note can be received as output of the machine-learned note prediction model based at least in part on the context vectors. The predicted visit note can include a set of predicted information expected to be included in a subsequently generated visit note associated with the first attendee.

FIELD

The present disclosure relates generally to machine-learned models forpredicting documentation of encounters between attendees.

BACKGROUND

Attendees at meetings or other encounters often generate records ornotes that document the meetings. Such records or notes can includeinformation relating to the information discussed in the meetings and/orinformation to be associated with future meetings. For instance, doctorsgenerally document doctor-patient encounters between the doctor and apatient by generating a visit note associated with the doctor-patientencounter. Such visit note can include information relating to a medicalhistory of the patient, discussions between the doctor and the patientduring the DPE (e.g. health problems or symptoms reported by thepatient), the doctor's findings (e.g. physical exam results), thedoctor's diagnosis, a proposed treatment plan, rationale for thetreatment plan, the doctor's analysis, care arrangements for thepatient, medical needs of the patient, non-medical needs of the patient,follow up procedures or plans (e.g. additional tests or studies, followup appointments, referrals to a specialist, etc.), and/or any othersuitable information relating to the DPE or to the patient in general.

Generating such notes can consume significant time and resources.Further, notes that are generated manually (e.g. typed, handwritten,etc.) can include errors. As an example, visit notes generated by adoctor that include such errors can lead to clinical errors that canendanger patient safety. Various tools and techniques have beenintroduced to aid in generating such notes, and to increase the accuracywith which the notes are recorded. Such conventional techniques forvisit note aid include providing templates that can be used to generatesuitable notes. Further techniques include auto-populating informationto be included in a visit note. However, such auto-population techniquescan be inaccurate, and/or inefficient.

SUMMARY

Aspects and advantages of embodiments of the present disclosure will beset forth in part in the following description, or may be learned fromthe description, or may be learned through practice of the embodiments.

One example aspect of the present disclosure is directed to acomputer-implemented method of predicting documentation associated withan encounter between attendees. The method includes obtaining, by one ormore computing devices, attendee data indicative of one or morepreviously generated visit notes associated with a first attendee of anencounter. The method further includes inputting, by the one or morecomputing devices, the attendee data into a machine-learned noteprediction model comprising a neural network. The method furtherincludes receiving as output of the machine-learned note predictionmodel, by the one or more computing devices, data indicative of apredicted visit note, the predicted visit note comprising a set ofpredicted information expected to be included in a subsequentlygenerated visit note associated with the first attendee.

Other example aspects of the present disclosure are directed to systems,apparatus, tangible, non-transitory computer-readable media, userinterfaces, memory devices, and electronic devices for predicting visitnotes.

These and other features, aspects and advantages of various embodimentswill become better understood with reference to the followingdescription and appended claims. The accompanying drawings, which areincorporated in and constitute a part of this specification, illustrateembodiments of the present disclosure and, together with thedescription, serve to explain the related principles.

BRIEF DESCRIPTION OF THE DRAWINGS

Detailed discussion of embodiments directed to one of ordinary skill inthe art are set forth in the specification, which makes reference to theappended figures, in which:

FIG. 1 depicts an example system according to example embodiments of thepresent disclosure;

FIGS. 2-3 depict example machine-learned note prediction modelsaccording to example embodiments of the present disclosure;

FIG. 4 depicts a flow diagram of an example method of predicting a visitnote according to example embodiments of the present disclosure;

FIG. 5 depicts a flow diagram of an example method of generating a visitnote according to example embodiments of the present disclosure; and

FIG. 6 depicts a flow diagram of an example method of training a noteprediction model according to example embodiments of the presentdisclosure.

DETAILED DESCRIPTION

Reference now will be made in detail to embodiments, one or moreexamples of which are illustrated in the drawings. Each example isprovided by way of explanation of the embodiments, not limitation of thepresent disclosure. In fact, it will be apparent to those skilled in theart that various modifications and variations can be made to theembodiments without departing from the scope or spirit of the presentdisclosure. For instance, features illustrated or described as part ofone embodiment can be used with another embodiment to yield a stillfurther embodiment. Thus, it is intended that aspects of the presentdisclosure cover such modifications and variations.

Example aspects of the present disclosure are directed tomachine-learned models configured to generate predicted documentationassociated with an encounter (e.g. meeting, appointment, interview,etc.) between two or more attendees. Such machine-learned models can beused to generate predicted visit notes relating to encounters betweenattendees. As used herein, the term “visit note” can refer to anysuitable documentation relating to an encounter between attendees. As anexample, an encounter can be a doctor's appointment, and attendees ofthe encounter can include a doctor and a patient. Doctors generallygenerate visit notes subsequent to a doctor-patient encounter (DPE),such as an appointment, check-up, physical, etc. A visit note candocument information relating to the patient and/or the DPE. Forinstance, the visit note can document information such as a medicalhistory of the patient, discussions between the doctor and the patientduring the DPE (e.g. health problems or symptoms reported by thepatient), the doctor's findings (e.g. physical exam results), thedoctor's diagnosis, a proposed treatment plan, rationale for thetreatment plan, the doctor's analysis, care arrangements for thepatient, medical needs of the patient, non-medical needs of the patient,follow up procedures or plans (e.g. additional tests or studies, followup appointments, referrals to a specialist, etc.), and/or any othersuitable information relating to the DPE or to the patient in general.Such visit notes may be generated by the doctor as a matter of coursesubsequent to a DPE and included in the medical records of the patient.

In this manner, example aspects of the present disclosure providesystems and methods that leverage machine learning to generate automatedpredicted visit notes associated with an encounter between a firstattendee (e.g. patient) and a second attendee (e.g. doctor). Althoughthe present disclosure is discussed with respect to generating predictedvisit notes associated with DPEs, it will be appreciated that exampleaspects of the present disclosure can be used in any suitable templatingapplication. For instance, the machine-learned models of the presentdisclosure can be trained to generate any suitable form of predictedtext output associated with any suitable encounter. In this manner,aspects of the present disclosure can be used to generate predicteddocumentation of encounters between attorneys and clients, employers andemployees, social workers and clients, interviewer and interviewee,and/or any other suitable encounter between two or more people.

More particularly, the systems and methods of the present disclosure caninclude and use a machine-learned note prediction model that is trainedto receive information associated with a first attendee associated witha subject encounter between the first attendee (e.g. patient) and asecond attendee (e.g. doctor), and to generate a predicted text outputdocumenting the encounter based at least in part on the informationassociated with the first attendee. The predicted text output caninclude predicted information that is expected to be included in asuitable document describing the encounter. In this manner, attendeedata associated with the first attendee can be obtained. For instance,in implementations wherein the encounter is a DPE between a doctor and apatient, the attendee data can include patient data associated with thepatient. The patient data can include any suitable patient data. Forinstance, such patient data can be indicative of one or more previousvisit notes respectively associated with one or more previous DPEs ofthe patient. As another example, the patient data can include anysuitable structured or unstructured data associated with a medicalhistory or record of the patient. As another example, the patient datacan include data obtained from one or more ambient and/or consumersensors. For instance, such patient data can include data obtained froma biometric sensor (e.g. heart rate data or other data) associated witha wearable computing device associated with the patient.

The patient data can be received subsequent to a provision of consentfrom the patient allowing the use of such patient data. In this manner,example aspects of the present disclosure can be performed contingent onreceiving such patient consent. In some implementations, the patientdata can be treated in one or more ways before or after it is used sothat personally identifiable information is removed or not storedpermanently.

In response to receiving the patient data, the note prediction model canoutput a predicted visit note associated with the patient. For instance,the predicted visit note can be associated with a subject DPE betweenthe doctor and the patient. A subject DPE can be a DPE occurring betweenthe doctor and the patient for which a visit note has not yet beengenerated. The predicted visit note can include information that isexpected to be included in a subsequent visit note associated with a DPEbetween the doctor and the patient that is generated by the doctor.

More particularly, the machine-learned note prediction model can includeone or more neural networks (e.g. deep neural networks). The neuralnetworks can be recurrent neural networks, such as long short-termmemory (LSTM) neural networks, gated recurrent unit (GRU) neuralnetworks, or other forms of neural networks. In one example a computingdevice can obtain patient data that describes one or more previous visitnotes associated with the patients. In some implementations, thecomputing device can further obtain patient data relating to a subjectDPE. The patient data relating to the subject DPE can be associated withinformation input by the doctor (or other person) relating to thesubject DPE. More particularly, the patient data relating to the subjectDPE can be associated with information that has not yet been recorded ina visit note. The computing device can input the patient data (e.g. thepatient data indicative of the one or more previous notes and/or thepatient data relating to the subject DPE) into the note predictionmodel, and can receive a predicted note as output of the note predictionmodel.

As indicated, the predicted visit note generated by the note predictionmodel can include predicted information expected to be included in asubsequent visit note associated with the patient. The predicted visitnote can include substantive information determined by the noteprediction model. For instance, such substantive information can includeinformation not included in the patient data. Such predicted informationcan be determined based at least in part on the patient data associatedwith the patient. Such predicted information can further be determinedbased at least in part on one or more previous visit notes generated bythe doctor (e.g. for any suitable number of patients), and/or one ormore previous visit notes generated by various other suitable doctorsfor various other suitable patients.

For instance, such predicted information can include informationrelating to a medical history of the patient, symptoms or healthproblems reported by the patient or expected to be reported by thepatient, a predicted diagnosis, a predicted treatment plan, a rationalefor the predicted treatment plan, predicted analysis, predicted carearrangements for the patient, predicted medical needs of the patient,predicted non-medical needs of the patient, predicted follow upprocedures or plans, and/or other suitable predicted informationrelating to the subject DPE or the patient in general. In this manner,the predicted visit note can include information learned by the noteprediction model based at least in part on the patient data, one or moreprevious visit notes generated by the doctor (e.g. for any suitablenumber of patients), one or more previous visit notes generated byvarious other suitable doctors for various other suitable patients,and/or other suitable structured or unstructured data associated withthe patient, one or more other patients, and/or the doctor. Inimplementations wherein the predicted visit note is determined based atleast in part on patient data input by the doctor associated with thesubject DPE, the predicted note can include information associated withsuch input patient data. For instance, if such patient data includes alist of symptoms provided by the patient (e.g. during the subject DPE orprior to the subject DPE, such as during a telephone call to schedule anappointment with the doctor), the predicted note can include the list ofsymptoms. In such implementations, the predicted information (e.g.predicted diagnosis, treatment plan, etc.) can be determined based atleast in part on the symptoms provided by the patient.

The predicted visit note can be generated such that the informationincluded in the predicted note is provided in a grammatically and/orsyntactically correct form. For instance, the note prediction model canbe configured to generate words, phrases, sentences, paragraphs,passages, etc. that collectively provide a coherent, easilyunderstandable, grammatically correct visit note that can be analyzed bythe doctor. For instance, in some implementations, the predicted notecan be synthesized by the note prediction model on a word-by-word basisto construct grammatically correct sentence structures that provide thepredicted information for a subsequent visit note.

In some implementations, the predicted visit note can be determinedbased at least in part on one or more previous visit notes generated bythe doctor. More particularly, the predicted visit note can be generatedsuch that the information provided in the predicted note is provided ina format and/or style determined based at least in part on the previousvisit notes generated by the doctor. For instance, the predicted visitnote can have a format (e.g. arrangement or configuration of informationincluded in the predicted note), style (e.g. writing style, such asvocabularies, sentence structure, paragraph structure, etc.) context,etc. that mimics or emulates that of one or more previous visit notesgenerated by the doctor. In this manner, the predicted notes can betailored to a particular format, stylization, etc. previously used bythe doctor.

As indicated, the note prediction model can include one or more neuralnetworks (e.g. feed-forward neural networks, recurrent neural networks).In some implementations, the note prediction model can include twophases. Each phase can include a neural network. In particular, in afirst phase, a computing device can input a set of patient data and/orother input data to a first neural network (e.g. LSTM network). Inresponse to receipt of the patient data, the first neural network canoutput a context vector. The context vector can specify informationincluded in the patient data. For instance, the context vector can begrammar agnostic, such that the context vector captures a general gistof the information included in the patient data, without regard togrammar or stylization of the information.

In a second phase of the note prediction model, the context vector canbe into a second neural network (e.g. LSTM network). The second neuralnetwork can output a predicted visit note associated with the patient inaccordance with example aspects of the present disclosure.

In some implementations, the second neural network can be configured tooutput a plurality of note prediction vectors. The note predictionvectors can describe information to potentially be included in asubsequent visit note. The note prediction vectors can be provided to asuggestion model configured to provide visit note suggestions to thedoctor (or other person) as the doctor (or other person) is generatingthe visit note. The suggestion model can be a machine-learned model thatincludes one or more hidden Markov models, support vector machines,neural networks (e.g., feed-forward neural networks, recurrent neuralnetworks such as LSTM recurrent neural networks), phrase-basedtranslators, sequence-to-sequence models, or other suitable models. Forinstance, the doctor can input a first text entry into a suitable userinterface. The first text entry can be provided as input to thesuggestion model along with the note prediction vectors, and thesuggestion model can provide as output one or more suggested textentries for completing the text entry. The doctor can then select one ormore of the suggested text entries to complete the text entry. In thismanner, the completed text entry can include information associated withthe first text entry provided by the user and the selected suggestedtext entry provided by the suggestion model.

The machine-learned models described herein can be trained usingsuitable training data, such as for instance, a global set of visitnotes generated by a plurality of doctors of a plurality of patients,and a set of doctor specific visit notes previously generated by thedoctor. More particularly, a training computing system can train thenote prediction models using a training dataset that includes a numberof global visit notes and a number of doctor specific visit notes.

In some implementations, to train the note prediction model, a firstsubset of a set of training data (e.g. data indicative of one or moreglobal visit notes and/or one or more doctor specific visit notes) isinput into the motion prediction model to be trained. In response toreceipt of such first portion, the motion prediction model outputs apredicted note that predicts a subsequent visit note based at least inpart on the first subset of training data. After such prediction, thetraining computing system can apply or otherwise determine a lossfunction that compares the predicted visit note output by the noteprediction model to an actual subsequent visit note of the training datathat the note prediction model attempted to predict. In someimplementations, the training computing system can backpropagate (e.g.,by performing truncated backpropagation through time) the loss functionthrough the note prediction model. In some implementations, the trainingcomputing system can determine the loss function for a whole mini-batchof training data at once.

In some implementations, in order to obtain the benefits of thetechniques described herein, users (e.g. encounter attendees, such aspatients, doctors, clients, employees, employers, etc.) may be requiredto allow the periodic collection and analysis of visit notes describingone or more encounters associated with the user. Therefore, users can beprovided with an opportunity to give consent as to whether and how muchthe systems and methods of the present disclosure collect and/or analyzesuch information. However, if the user does not allow collection and useof such information, then the user may not receive the benefits of thetechniques described herein. In addition, in some embodiments, certaininformation or data can be treated in one or more ways before or afterit is used, so that personally identifiable information is removed ornot stored permanently.

With reference now to the figures, example aspects of the presentdisclosure will be discussed in greater detail. For instance, FIG. 1depicts an example system 100 for predicting visit notes according toexample aspects of the present disclosure. The system 100 includes auser computing device 102, a server computing system 130, and a trainingcomputing system 150 that are communicatively coupled over a network180.

The user computing device 102 can be any type of computing device, suchas, for example, a personal computing device (e.g., laptop or desktop),a mobile computing device (e.g., smartphone or tablet), a gaming consoleor controller, a wearable computing device, an embedded computingdevice, or any other type of computing device.

The user computing device 102 includes one or more processors 112 and amemory 114. The one or more processors 112 can be any suitableprocessing device (e.g., a processor core, a microprocessor, an ASIC, aFPGA, a controller, a microcontroller, etc.) and can be one processor ora plurality of processors that are operatively connected. The memory 114can include one or more non-transitory computer-readable storagemediums, such as RAM, ROM, EEPROM, EPROM, flash memory devices, magneticdisks, etc., and combinations thereof. The memory 114 can store data 116and instructions 118 which are executed by the processor 112 to causethe user computing device 102 to perform operations.

The user computing device 102 can store or include one or more noteprediction models 120. For example, the note prediction models 120 canbe or can otherwise include various machine-learned models such asneural networks (e.g., deep neural networks) or other multi-layernon-linear models. Neural networks can include recurrent neural networks(e.g., long short-term memory recurrent neural networks), feed-forwardneural networks, or other forms of neural networks. Example noteprediction models 120 are discussed with reference to FIGS. 2 and 3.

In some implementations, the one or more note prediction models 120 canbe received from the server computing system 130 over network 180,stored in the user computing device memory 114, and the used orotherwise implemented by the one or more processors 112. In someimplementations, the user computing device 102 can implement multipleparallel instances of a single note prediction model 120.

More particularly, the note prediction model(s) 120 can be used intemplating a predicted visit note associated with a patient and/or adoctor. For instance, the predicted visit note can be associated with asubject DPE between the doctor and the patient. The subject DPE can bean upcoming DPE (e.g. a scheduled DPE) or a DPE that has alreadyoccurred. In this manner, the note prediction model(s) 120 can generatethe predicted visit note for review by the doctor or other person. Thenote prediction model(s) 120 can be trained to determine a predictedvisit note for a patient. More particularly, the user computing device102 can provide patient data as input to the note prediction model(s)120, and the not prediction model(s) 120 can provide a predicted visitnote as output based at least in part on the patient data. The patientdata can include data indicative of one or more previous visit notesassociated with one or more previous DPEs between the patient and one ormore doctors. The patient data can further include data associated withthe subject DPE. For instance, such data can include any suitable dataassociated with the patient recorded in association with the subject DPE(either prior to the subject DPE or subsequent to the subject DPE). Suchdata can be input to the user computing device 102 or other computingdevice by any suitable user.

The note prediction model(s) 120 can be employed within various suitableapplications associated with the user computing device 102. Forinstance, the note prediction model(s) 120 can be employed within thecontext of a suitable application associated with the templating ofvisit notes. As another example, the note prediction model(s) 120 can beincluded as a browser plug-in or web-based application associated withthe user device 102.

Additionally or alternatively, one or more note prediction models 140can be included in or otherwise stored and implemented by the servercomputing system 130 that communicates with the user computing device102 according to a client-server relationship. For example, the noteprediction models 140 can be implemented by the server computing system140 as a portion of a web service (e.g., a visit note templatingservice). Thus, one or more models 120 can be stored and implemented atthe user computing device 102 and/or one or more models 140 can bestored and implemented at the server computing system 130.

The user computing device 102 can also include one or more user inputcomponent 122 that receives user input. For example, the user inputcomponent 122 can be a touch-sensitive component (e.g., atouch-sensitive display screen or a touch pad) that is sensitive to thetouch of a user input object (e.g., a finger or a stylus). Thetouch-sensitive component can serve to implement a virtual keyboard.Other example user input components include a microphone suitable forvoice recognition, a traditional keyboard, mouse or other means by whicha user can enter a communication.

The server computing system 130 includes one or more processors 132 anda memory 134. The one or more processors 132 can be any suitableprocessing device (e.g., a processor core, a microprocessor, an ASIC, aFPGA, a controller, a microcontroller, etc.) and can be one processor ora plurality of processors that are operatively connected. The memory 134can include one or more non-transitory computer-readable storagemediums, such as RAM, ROM, EEPROM, EPROM, flash memory devices, magneticdisks, etc., and combinations thereof. The memory 134 can store data 136and instructions 138 which are executed by the processor 132 to causethe server computing system 130 to perform operations.

In some implementations, the server computing system 130 includes or isotherwise implemented by one or more server computing devices. Ininstances in which the server computing system 130 includes pluralserver computing devices, such server computing devices can operateaccording to sequential computing architectures, parallel computingarchitectures, or some combination thereof.

As described above, the server computing system 130 can store orotherwise includes one or more machine-learned note prediction models140. For example, the note prediction model(s) 140 can be or canotherwise include various machine-learned models such as neural networks(e.g., deep recurrent neural networks) or other multi-layer non-linearmodels. Example communication assistance models 140 are discussed withreference to FIGS. 2 and 3.

The server computing system 130 can train the communication assistancemodels 140 via interaction with the training computing system 150 thatis communicatively coupled over the network 180. The training computingsystem 150 can be separate from the server computing system 130 or canbe a portion of the server computing system 130.

The training computing system 150 includes one or more processors 152and a memory 154. The one or more processors 152 can be any suitableprocessing device (e.g., a processor core, a microprocessor, an ASIC, aFPGA, a controller, a microcontroller, etc.) and can be one processor ora plurality of processors that are operatively connected. The memory 154can include one or more non-transitory computer-readable storagemediums, such as RAM, ROM, EEPROM, EPROM, flash memory devices, magneticdisks, etc., and combinations thereof. The memory 154 can store data 156and instructions 158 which are executed by the processor 152 to causethe training computing system 150 to perform operations. In someimplementations, the training computing system 150 includes or isotherwise implemented by one or more server computing devices.

The training computing system 150 can include a model trainer 160 thattrains the machine-learned models 140 stored at the server computingsystem 130 using various training or learning techniques, such as, forexample, backwards propagation of errors. In some implementations,performing backwards propagation of errors can include performingtruncated backpropagation through time. The model trainer 160 canperform a number of generalization techniques (e.g., weight decays,dropouts, etc.) to improve the generalization capability of the modelsbeing trained.

In particular, the model trainer 160 can train a note prediction model140 based on a set of training data 142. The training data 142 caninclude, for example, data indicative of a plurality of global visitnotes associated with a plurality of DPEs between a plurality of doctorsand a plurality of patients. The training data 142 can further include aplurality of doctor specific visit notes generated by the doctorassociated with a plurality of DPEs between the doctor and a pluralityof patients. In some implementations, the global visit notes can begrouped according to patient, doctor, or other suitable grouping metricassociated with the respective visit notes. For instance, a group ofglobal visit notes can be grouped in a sequence according to achronology of DPEs associated with a patient. Similarly, the doctorspecific notes can be grouped according to patient or other suitablegrouping metrics in accordance with a chronology of DPEs.

In some implementations, if the user has provided consent, the trainingexamples can be provided by the user computing device 102 (e.g., basedon visit notes previously provided by the user of the user computingdevice 102). Thus, in such implementations, the model 120 provided tothe user computing device 102 can be trained by the training computingsystem 150 on user-specific communication data received from the usercomputing device 102. In some instances, this process can be referred toas personalizing the model.

The model trainer 160 includes computer logic utilized to providedesired functionality. The model trainer 160 can be implemented inhardware, firmware, and/or software controlling a general purposeprocessor. For example, in some implementations, the model trainer 160includes program files stored on a storage device, loaded into a memoryand executed by one or more processors. In other implementations, themodel trainer 160 includes one or more sets of computer-executableinstructions that are stored in a tangible computer-readable storagemedium such as RAM hard disk or optical or magnetic media.

The network 180 can be any type of communications network, such as alocal area network (e.g., intranet), wide area network (e.g., Internet),or some combination thereof and can include any number of wired orwireless links. In general, communication over the network 180 can becarried via any type of wired and/or wireless connection, using a widevariety of communication protocols (e.g., TCP/IP, HTTP, SMTP, FTP),encodings or formats (e.g., HTML, XML), and/or protection schemes (e.g.,VPN, secure HTTP, SSL).

FIG. 1 illustrates one example computing system that can be used toimplement the present disclosure. Other computing systems can be used aswell. For example, in some implementations, the user computing device102 can include the model trainer 160 and the training dataset 162. Insuch implementations, the communication assistance models 120 can beboth trained and used locally at the user computing device 102. In someof such implementations, the user computing device 102 can implement themodel trainer 160 to personalize the communication assistance models 120based on user-specific data.

FIG. 2 depicts a block diagram of an example note prediction model 200according to example embodiments of the present disclosure. In someimplementations, the note prediction model 200 can be configured toreceive a set of patient data descriptive of one or more visit notesgenerated for one or more previous DPEs and/or information associatedwith a subject DPE associated with a patient. The note prediction modelcan be trained to output one or more predicted visit notes in responseto the receipt of the patient data. In some implementations, the set ofpatient data can be a set of sequential patient data. For instance, thesequential patient data can describe visit notes in an order in whichthey were generated (e.g. corresponding to a chronological order of theDPEs).

The note prediction model 200 includes an interpreter 202 and apredictor 204. The interpreter 202 can include one or more neuralnetworks (e.g. deep neural networks). For instance, the interpreter 202can include one or more recurrent neural networks, such as for instance,LSTM recurrent neural networks, gated recurrent unit networks, or othersuitable networks. In other implementations, in addition oralternatively to one or more neural networks, the interpreter 202 of thenote prediction model 200 can include other forms of machine-learnedmodels such as, for example, support vector machines, hidden Markovmodels, and other machine-learned classifiers. In addition, in someimplementations, the interpreter 202 can include or implement additionalalternative approaches such as rule-based systems (e.g., asmachine-learned from data or manually created).

The one or more neural networks or other machine-learned models of theinterpreter 202 can be trained on a training dataset that includes, forexample, training visit notes (e.g. global visit notes and/or doctorspecific visit notes). The training dataset can include any suitablestructured or unstructured data or information relating to the medicalhistory or record of the user. In some implementations, the trainingdata can include data obtained from one or more ambient and/or consumersensors. For instance, such training data can include data obtained froma biometric sensor associated with a wearable computing deviceassociated with the patient. As indicated, in some implementations, suchtraining data can be received contingent upon receiving consent from therespective patients with which the training data are associated. In thismanner, such patients can be provided with opportunities to controlwhether such data is collected and/or the manner in which the data isanalyzed. As indicated, the training data can include a sequential setof visit notes ordered chronologically according to DPE. In someimplementations the training data can be grouped by doctor and/or bypatient. For instance, a subset of training data can include dataindicative of a plurality of chronologically ordered visit notesgenerated for a patient (e.g. by one or more doctors) in response to aplurality of DPEs. As another example, a subset of training data caninclude data indicative of a plurality of (e.g. chronologically ordered)visit notes generated by a particular doctor.

In this manner, the interpreter 202 can be trained to recognize and/orinterpret information included within patient data (e.g. visit notes orother suitable communications) for a patient. For instance, theinterpreter 202 can be trained to output one or more context vectorsdescriptive of one or more input visit notes provided as input to theinterpreter 202. The context vectors can specify information includedwithin the visit notes and/or other patient data. In someimplementations, the context vectors can specify such information in agrammar and/or formatting agnostic manner. A context vector can providea general gist of information included in at least a portion of one ormore visit notes.

The note prediction model 200 can further include predictor 204. Thepredictor 204 can include one or more neural networks (e.g. deep neuralnetworks). For instance, the predictor 204 can include one or morerecurrent neural networks, such as for instance, LSTM recurrent neuralnetworks, gated recurrent unit networks, or other suitable networks. Inother implementations, in addition or alternatively to one or moreneural networks, the interpreter 242 of the note prediction model 200can include other forms of machine-learned models such as, for example,support vector machines, hidden Markov models, and other machine-learnedclassifiers. In addition, in some implementations, the interpreter 202can include or implement additional alternative approaches such asrule-based systems (e.g., as machine-learned from data or manuallycreated).

The output of the interpreter 202 can be provided to the predictor 204.The one or more neural networks or other machine-learned models of thepredictor 204 can be trained on a suitable training data set accordingto example aspects of the present disclosure. For instance, the trainingdata set can be the same data set or a similar data set as the data setused to train the interpreter 202. In this manner, the training data setcan include data indicative of one or more global visit notes and/or oneor more doctor specific visit notes.

The predictor 204 can be trained to output one or more predicted visitnotes as a result of receiving the context vectors or other output ofthe interpreter 202. In this manner, the predictor 204 can output apredicted visit note based at least in part on the patient data providedto the interpreter 202. In some implementations, the predictor 204 canbe configured to generate a predicted visit note that predictsinformation that would be included in a subsequent visit note generatedfor the user (e.g. for the subject DPE). As indicated, such predictedinformation can include information relating to medical history,predicted symptoms, predicted treatment plans, predicted exam results,predicted discussions, etc. that are expected to be included in asubsequent visit note for the patient.

In some implementations, the predictor 204 can be configured to generatea predicted visit note that is formatted in accordance with one or moreprevious visit notes generated by a particular doctor. For instance, inimplementations wherein the note prediction model 200 is trained ontraining data associated with a plurality of previous visit notesgenerated by the doctor (e.g. wherein the training is personalizedtowards the doctor), the predictor 204 can be trained to learn andpredict a suitable formatting, stylization, grammar, etc. used by thedoctor in generating such previous visit notes. The predictor 204 canthen be trained to output predicted visit notes in accordance with suchlearned formatting, stylization, grammar, etc. In this manner, thepredictor 204 can be trained to output predicted visit notes thatinclude predicted information formatted and written in accordance withthe doctor's previously generated visit notes.

The predicted visit notes can be output on a display of a user computingdevice associated with the doctor (e.g. user computing device 102depicted in FIG. 1). The doctor or other suitable person can review thepredicted visit note, and can make any desired changes or additions tothe visit note through an interaction with the user device. In thismanner, the predicted visit note can be presented as a baseline ortemplate visit note associated with a subject DPE.

FIG. 3 depicts a block diagram of an example note prediction model 210according to example embodiments of the present disclosure. The noteprediction model 210 is similar to the prediction model 200 depicted inFIG. 2, except the model 210 includes a suggestion model 216. Moreparticularly, the note prediction model 210 includes an interpreter 212and a predictor 214. The interpreter 212 can be configured and trainedto receive patient data associated with a patient as input and toprovide one or more context vectors associated with the patient data asoutput. The predictor 214 can be configured and trained to receive thecontext vectors as input, and to generate one or more prediction vectorsas output. The prediction vectors can be descriptive of information topotentially be included in a subsequent visit note associated with thepatient. For instance, the prediction vectors can be descriptive ofpredicted information (e.g. medical history, predicted symptoms,predicted treatment plan, etc.) associated with the subsequent visitnote.

The prediction vectors can be provided as input to the suggestion model216. The suggestion model 216 can include one or more neural networks(e.g. deep neural networks). For instance, the suggestion model 216 caninclude one or more recurrent neural networks, such as for instance,LSTM recurrent neural networks, gated recurrent unit networks, or othersuitable networks. In other implementations, in addition oralternatively to one or more neural networks, the suggestion model 216of the note prediction model 210 can include other forms ofmachine-learned models such as, for example, support vector machines,hidden Markov models, and other machine-learned classifiers. Inaddition, in some implementations, the note prediction model 216 caninclude or implement additional alternative approaches such asrule-based systems (e.g., as machine-learned from data or manuallycreated).

The suggestion model 216 can be configured to receive the predictionvectors from the predictor 214 as input. In some implementations, thesuggestion model 216 can be configured to receive one or more textentries input by a user, for instance, through an interaction with theuser computing device associated with the note prediction model 210. Thetext entries can include one or more characters, symbols, words,phrases, sentences, etc. For instance, the text entries can beincomplete words, sentences, phrases, etc. input by the user. In thismanner, the text entries can be provided to the suggestion model 216 inreal time or near real time as the user inputs the text entries. Forinstance, the text entries can be provided to the suggestion model 216sequentially (e.g. on a character-by-character basis, word-by-wordbasis, etc.) as the user inputs the entries. As the suggestion model 216receives the text entries, the suggestion model 216 can analyze the textentries in view of the prediction vectors to determine one or moresuggested text entries based at least in part on the prediction vectors.More particularly, the one or more suggested text entries can bedetermined such that the suggested text entries can be used to completethe text entry provided by the user.

The one or more suggested text entries can be provided to the user, forinstance, in the user interface with which the user is interacting (e.g.to input the text entries). The user can select a suggested text entryas desired to facilitate a determination of a completed text entry bythe user computing device. For instance, the completed text entry can bea combination of the text entry input by the user and the selectedsuggested text entry. As another example, the completed text entry canbe a new text entry determined based at least in part on the input textentry and the selected suggested text entry to capture the informationincluded in the input and selected suggested text entries.

FIG. 4 depicts a flow diagram of an example method (300) of predictingvisit notes according to example aspects of the present disclosure.Method (300) can be implemented by one or more computing devices, suchas one or more of the computing devices depicted in FIG. 1. In addition,FIG. 4 depicts steps performed in a particular order for purposes ofillustration and discussion. Those of ordinary skill in the art, usingthe disclosures provided herein, will understand that the steps of anyof the methods discussed herein can be adapted, rearranged, expanded,omitted, or modified in various ways without deviating from the scope ofthe present disclosure.

At (302), the method (300) can include obtaining a set of attendee dataassociated with a first attendee of a subject encounter between thefirst attendee and a second attendee. For instance, the first attendeecan be a patient associated with a DPE and the second attendee can be adoctor associated with the DPE. In such instances, the attendee data caninclude patient data associated with the patient. The patient data caninclude data descriptive of one or more visit notes previously generatedfor the patient. Each visit note can be associated with a particular DPEbetween the patient and a doctor. In some implementations, the patientdata can be arranged sequentially based at least in part on an order ofgeneration of the visit notes. For instance, data descriptive of a firstvisit note generated at a first time can be arranged prior to datadescriptive of a second visit note generated at a subsequent timerelative to the first time. In some implementations, the patient datacan further include data associated with a subject DPE between thepatient and a doctor.

At (304), the method (300) can include providing the attendee data asinput into a machine-learned note prediction model. For instance, a usercomputing device can input the set of patient data into a localmachine-learned note prediction model. As another example, a usercomputing device can transmit patient data over a network to a servercomputing device, and the server computing device can input the patientdata into a machine-learned note prediction model stored at the servercomputing device. In implementation wherein the note prediction model isstored at a server computing device, the patient data may be stored atthe server computing device or other suitable location. For instance,data indicative of one or more visit notes may be stored at the servercomputing device and/or at a user computing device communicativelycoupled to the server computing device.

In some implementations, providing the attendee data as input to thenote prediction model can include providing a string of text for inputinto the note prediction model. For instance, patient data associatedwith a first visit note may be provided as a first string of textaccording to an order in which the first visit note is to be read.Patient data associated with a second visit note can then be provided tothe note prediction model as a second string of text according to anorder in which the second visit note is to be read. As indicated, thepatient data can further be provided to the note prediction modelaccording to an order in which the visit notes were generated. It willbe appreciated that the patient data can be provided to the noteprediction model in any suitable manner without deviating from the scopeof the present disclosure.

In some implementations, other suitable data can be provided to the noteprediction model as input. For instance, such other data can includedata associated with a global set of visit notes associated with aplurality of patients, a set of doctor specific visit notes generated bythe doctor involved in the DPE, other suitable structured orunstructured information associated with the plurality of patientsand/or the doctor, or other data. In this manner, it will be appreciatedthat any suitable information can be provided as input to the noteprediction model to generate a predicted visit note.

At (306), the method (300) can include receiving data indicative of apredicted visit note associated with the first attendee and the secondattendee as output of the note-prediction model. The predicted visitnote can include predicted information expected to be included in asubsequent visit note for the first attendee based at least in part onthe input attendee data (e.g. the previous visit notes for the patient).In some implementations, the predicted visit note can include suchpredicted information in a format, stylization, grammar, etc. that istailored to that used by the second attendee (e.g. the doctor). Forinstance, the predicted visit note can be generated based at least inpart on one or more previous visit notes generated by the doctor for oneor more patients. In this manner, the note prediction model can betrained on a suitable set of training to output a suitable predictionnote. The training of the note prediction model will be discussed ingreater detail with respect to FIG. 6.

At (308), the method (300) can include providing the predicted visitnote for presentation in a user interface. For instance, the predictedvisit note can be provided for display in a user interface of a usercomputing device associated with the second attendee. In this manner,the predicted visit note can be presented to the second attendee (orother suitable person) as a baseline or template visit note for asubject encounter between the first attendee and the second attendee.The second attendee can edit, rearrange, add to, etc. the presentedpredicted visit note as desired through one or more suitableinteractions with the user computing device. Once the visit noted isacceptable to the second attendee, the finalized visit note can bestored, for instance as part of the first attendee's and/or the secondattendee's records for future use.

FIG. 5 depicts a flow diagram of an example method (400) of generatingvisit notes according to example embodiments of the present disclosure.Method (300) can be implemented by one or more computing devices, suchas one or more of the computing devices depicted in FIG. 1. In addition,FIG. 5 depicts steps performed in a particular order for purposes ofillustration and discussion.

At (402), the method (400) can include receiving one or more predictionvectors as input by a machine-learned suggestion model in accordancewith example aspects of the present disclosure. For instance, in someimplementations, a predictor portion of the machine-learned noteprediction model described with respect to FIG. 4 can generate one ormore prediction vectors indicative of potential information that may beincluded within a predicted visit note. Each prediction vector can bedescriptive of a subset of information to be potentially included in apredicted visit note.

At (404), the method (400) can include receiving data indicative of afirst text entry input by a user of a user computing device. Forinstance, the user can input the first text entry through suitableinteraction with the user computing device. The data indicative of thefirst text entry can be associated with a string of text, and can bereceived, for instance, in accordance with an order in which the text isinput by the user. In this manner, the data indicative of the first textentry can be received in concurrently (or near concurrently) with theinputting of the text entry by the user.

At (406), the method (400) can include determining one or more suggestedtext entries based at least in part on the first text entry and the oneor more prediction vectors. For instance, the suggested text entries canbe text entries that can complete the first text entry to providesuitable information associated with the prediction vectors. In thismanner, the suggested text entries can be determined to construct agrammatically, stylistically suitable, sentence, phrase, etc. thatincludes suitable information to be included in the predicted visitnote. In this manner, such suggested text entries can be determinedbased at least in part on the patient data, one or more global visitnotes, and one or more doctor specific visit notes.

At (408), the method (400) can include receiving data indicative of auser input associated with a selected suggested text entry. Forinstance, one or more suggested text entries can be presented to theuser in a user interface associated with the user computing device. Thepresented suggested text entries can be interactive such that the usercan select one or more of the suggested text entries to facilitate aconstruction of a completed text entry. In this manner, the selectedsuggested text entry can be a suggested text entry that was selected bythe user.

At (410), the method (400) can include generating a completed text entrybased at least in part on the first text entry and the one or moreselected suggested text entries. For instance, a completed text entrycan be a combination of the first text entry and a selected suggestedtext entry. In some implementations, the completed text entry can be anew text entry that captures the information included in the first textentry and the selected suggested text entry.

FIG. 6 depicts a flow diagram of an example method (500) of training anote prediction model according to example aspects of the presentdisclosure. Method (500) can be implemented by one or more computingdevices, such as one or more of the computing devices depicted inFIG. 1. In addition, FIG. 6 depicts steps performed in a particularorder for purposes of illustration and discussion.

At (502), the method (500) can include inputting a first subset of a setof training data into a note prediction model according to exampleaspects of the present disclosure. The training data can include dataindicative of a plurality of global visit notes previously generated byany suitable number of doctors for any suitable number of patients. Insome implementations the note prediction model can be personalized for aparticular doctor. In such implementations, the training data caninclude data indicative of a plurality of doctor specific visit notespreviously generated by the doctor for any suitable number of patients.It will be appreciated that various other suitable training data can beused without deviating from the scope of the present disclosure.

In this manner, the first subset of training data can include dataindicative of one or more global visit notes and/or data indicative ofone or more doctor specific visit notes. As indicated, the first subsetof training data can be provided as one or more strings of text inaccordance with an order in which the visit notes are to be read.Further, the first subset of training data can be provided in accordancewith an order in which the visit notes were generated.

At (504), the method (500) can include receiving an output from the noteprediction model. In some implementations, the output can include apredicted note. In some implementations, the output can include one ormore prediction vectors.

In some implementations, the note prediction model can include varioussuitable portions (e.g. interpreter 202, predictor 204, suggestion model216, etc.). In such implementations, the portions of note predictionmodel can be trained independently in portion-specific trainingsessions. For instance, the interpreter can be trained independently ofa training of the predictor. In such implementations, the receivedoutput from the note prediction model can be an intermediate outputgenerated by the particular portion associated with the trainingsession. For instance, the received output can be one or more contextvectors associated with the interpreter, a predicted visit note or oneor more prediction vectors generated by the predictor, one or moresuggested text entries generated by the suggestion model, etc.

At (506), the method (500) can include determining a loss function thatdescribes a difference between the output of the note prediction modeland a second subset of the training data. For instance, the secondsubset of training data can include a subsequent visit note relative tothe visit notes included within the first subset of training data. Thesubsequent visit note can be a visit note generated by a particularpatient and/or generated by a particular doctor. The subsequent visitnote can be the visit note that the note prediction model aims togenerate as a result of receiving the first subset of training data. Inthis manner, the loss function can describe a difference between apredicted visit note generated by the note prediction model and thesubsequent visit note.

At (508), the method (500) can include training the note predictionmodel based at least in part on the loss function. For instance,training the note prediction model can include performing truncatedbackwards propagation through time to backpropagate the loss functionthrough the note prediction model. A number of generalization techniques(e.g., weight decays, dropouts, etc.) can optionally be performed at 508to improve the generalization capability of the models being trained.After the model has been trained at 508, it can be provided to andstored at a user computing device for use in predicting visit notes atthe user computing device. More particularly, the training proceduredescribed in 502-508 can be repeated several times (e.g., until anobjective loss function no longer improves) to train the model.

The technology discussed herein makes reference to servers, databases,software applications, and other computer-based systems, as well asactions taken and information sent to and from such systems. One ofordinary skill in the art will recognize that the inherent flexibilityof computer-based systems allows for a great variety of possibleconfigurations, combinations, and divisions of tasks and functionalitybetween and among components. For instance, server processes discussedherein may be implemented using a single server or multiple serversworking in combination. Databases and applications may be implemented ona single system or distributed across multiple systems. Distributedcomponents may operate sequentially or in parallel.

While the present subject matter has been described in detail withrespect to specific example embodiments thereof, it will be appreciatedthat those skilled in the art, upon attaining an understanding of theforegoing may readily produce alterations to, variations of, andequivalents to such embodiments. Accordingly, the scope of the presentdisclosure is by way of example rather than by way of limitation, andthe subject disclosure does not preclude inclusion of suchmodifications, variations and/or additions to the present subject matteras would be readily apparent to one of ordinary skill in the art.

What is claimed is:
 1. A computer-implemented method of predictingdocumentation associated with an encounter between attendees, the methodcomprising: obtaining, by one or more computing devices, attendee dataindicative of one or more previously generated visit notes associatedwith a first attendee of a subject encounter between the first attendeeand a second attendee; inputting, by the one or more computing devices,the attendee data into a machine-learned note prediction modelcomprising a first neural network and a second neural network;receiving, by the one or more computing devices, one or more contextvectors as output of the first neural network; inputting, by the one ormore computing devices, the one or more context vectors into the secondneural network of the machine-learned note prediction model; receiving,by the one or more computing devices, one or more prediction vectors asoutput of the second neural network, the one or more prediction vectorsbeing descriptive of information to potentially be included in apredicted visit note; providing, by the one or more computing devices,the one or more prediction vectors as input to a suggestion model of thenote prediction model; providing, by the one or more computing devices,data indicative of a first text entry input by a user as input to thesuggestion model; and receiving as output of the machine-learned noteprediction model, by the one or more computing devices, data indicativeof a predicted visit note, the predicted visit note comprising a set ofpredicted information expected to be included in a subsequentlygenerated visit note associated with the first attendee, the set ofpredicted information comprising one or more suggested text entriesdetermined based at least in part on the one or more prediction vectorsand the data indicative of the first text entry.
 2. Thecomputer-implemented method of claim 1, wherein the attendee datacomprises data indicative of one or more previously generated visitnotes for the first attendee, each previously generated visit note beingassociated with a previous encounter of the first attendee.
 3. Thecomputer-implemented method of claim 1, wherein the attendee datafurther comprises data associated with the subject encounter between thefirst attendee and the second attendee.
 4. The computer-implementedmethod of claim 3, wherein the data associated with the subjectencounter comprises data provided to a user computing device prior to ageneration of a visit note associated with the subject encounter.
 5. Thecomputer-implemented method of claim 1, wherein the first attendee is apatient associated with the subject encounter and the second attendee isa doctor associated the subject encounter, and wherein the attendee dataincludes data relating to a medical history of the patient.
 6. Thecomputer-implemented method of claim 5, wherein the predictedinformation comprises substantive information expected to be included ina subsequently generated visit note associated with the subjectencounter based at least in part on the attendee data.
 7. Thecomputer-implemented method of claim 1, wherein the first neural networkcomprises a long short-term memory recurrent neural network.
 8. Thecomputer-implemented method of claim 1, wherein receiving, by the one ormore computing devices, data indicative of a predicted visit notecomprises receiving the data indicative of the predicted visit note asan output of the second neural network.
 9. The computer-implementedmethod of claim 1, further comprising training, by the one or morecomputing devices, the note prediction model based on a set of trainingdata; wherein training, by the one or more computing devices, the noteprediction model comprises backpropagating, by the one or more computingdevices, a loss function through the note prediction model.
 10. Thecomputer-implemented method of claim 9, wherein the training datacomprises data indicative of a plurality of global visit notes.
 11. Thecomputer-implemented method of claim 9, wherein the training datacomprises data indicative of a plurality of doctor specific visit notes.12. A computing system, comprising: one or more processors; and one ormore memory devices, the one or more memory devices storingcomputer-readable instructions that when executed by the one or moreprocessors cause the one or more processors to perform operations, theoperations comprising: obtaining attendee data indicative of one or moreprevious visit notes associated with a first attendee of a subjectencounter between the first attendee and a second attendee; inputtingthe attendee data into a machine-learned note prediction modelcomprising a first neural network and a second neural network;receiving, by the one or more computing devices, one or more contextvectors as output of the first neural network; inputting, by the one ormore computing devices, the one or more context vectors into the secondneural network of the machine-learned note prediction model; receiving,by the one or more computing devices, one or more prediction vectors asoutput of the second neural network, the one or more prediction vectorsbeing descriptive of information to potentially be included in apredicted visit note; providing, by the one or more computing devices,the one or more prediction vectors as input to a suggestion model of thenote prediction model; providing, by the one or more computing devices,data indicative of a first text entry input by a user as input to thesuggestion model; and receiving, as output of the machine-learned noteprediction model, data indicative of a predicted visit note, thepredicted visit note comprising a set of predicted information expectedto be included in a subsequently generated visit note associated withthe subject encounter, the set of predicted information comprising oneor more suggested text entries determined based at least in part on theone or more prediction vectors and the data indicative of the first textentry.
 13. The computing system of claim 12, wherein the attendee datacomprises data indicative of one or more previously generated visitnotes for the first attendee, each previously generated visit note beingassociated with a previous encounter of the first attendee.
 14. One ormore tangible, non-transitory computer-readable media storingcomputer-readable instructions that when executed by one or moreprocessors cause the one or more processors to perform operations, theoperations comprising obtaining patient data indicative of one or moreprevious visit notes associated with a patient; inputting the patientdata into a first neural network associated with a machine-learned noteprediction model; receiving one or more context vectors as output of thefirst neural network; inputting the one or more context vectors into asecond neural network associated with the machine-learned noteprediction model; receiving one or more prediction vectors as output ofthe second neural network, the one or more prediction vectors beingdescriptive of information to potentially be included in a predictedvisit note; providing the one or more prediction vectors as input to asuggestion model associated with the note prediction model; providingdata indicative of a first text entry input by a user as input to thesuggestion model; and receiving data indicative of a predicted visitnote, the predicted visit note comprising a set of predicted informationexpected to be included in a subsequently generated visit noteassociated with the patient, the set of predicted information comprisingone or more suggested text entries determined based at least in part onthe one or more prediction vectors and the data indicative of the firsttext entry.